Skip to content

Add Yaml include list (Discussion #2817)#2887

Closed
christiangoerdes wants to merge 16 commits intomasterfrom
yaml-include-list
Closed

Add Yaml include list (Discussion #2817)#2887
christiangoerdes wants to merge 16 commits intomasterfrom
yaml-include-list

Conversation

@christiangoerdes
Copy link
Copy Markdown
Collaborator

@christiangoerdes christiangoerdes commented Mar 19, 2026

Summary by CodeRabbit

  • New Features

    • Support for include directives in YAML configs with recursive, relative resolution; directory includes load *.apis.yaml/*.apis.yml; cyclic includes are detected and rejected.
    • Added a top-level include list element (generated) to simplify multi-file inclusion.
  • Documentation

    • Stable metadata IDs added for header-filter include/exclude rules to improve tooling and schema generation.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 19, 2026

Caution

Review failed

An error occurred during the review process. Please try again later.

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Parses YAML in two phases: collects include vs config snippets, recursively resolves includes (files or directories matching .apis.yaml|.apis.yml) with path normalization and cycle detection, shares a global bean index across recursion; adds IncludeList generator and integrates it into annotation processing; tweaks header-filter annotation metadata.

Changes

Cohort / File(s) Summary
YAML Parsing & Include Handling
annot/src/main/java/com/predic8/membrane/annot/yaml/GenericYamlParser.java
Rewrote parser to collect include/config snippets, expand includes recursively (file/dir resolution, *.apis.yaml/*.apis.yml filtering), normalize paths, read included files via Files.readString, detect/report cyclic includes using an include stack, and use a shared beanIndex. Added helper records and IO/path utilities; removed unused import.
Annotation-processor: IncludeList Generator
annot/src/main/java/com/predic8/membrane/annot/generator/IncludeListClassGenerator.java, annot/src/main/java/com/predic8/membrane/annot/SpringConfigurationXSDGeneratingAnnotationProcessor.java
New generator IncludeListClassGenerator emits a public IncludeList class (mutable List<String> includes, setter/getter, docs describing recursive include semantics). Processor invokes this generator and returns early when it writes classes to trigger reprocessing.
Metadata tweak: Header Filter
core/src/main/java/com/predic8/membrane/core/interceptor/headerfilter/HeaderFilterInterceptor.java
Updated @MCElement on static Include/Exclude rule classes to set component=false and explicit id values ("headerFilter-include", "headerFilter-exclude"); no runtime logic changes.

Sequence Diagram

sequenceDiagram
    participant User
    participant Parser as YAML Parser
    participant FS as File System
    participant Stack as Include Stack
    participant Beans as Bean Builder

    User->>Parser: parse(grammar, yaml)
    Parser->>Parser: split documents -> includeSnippets + configSnippets
    Parser->>Stack: create IncludeContext(basePath)
    loop each includeSnippet / entry
        Parser->>Stack: normalize & checkCycle(path)
        alt cycle detected
            Stack-->>Parser: throw ConfigurationParsingException
        else
            Parser->>FS: resolve path (file or directory)
            alt directory
                FS-->>Parser: list `*.apis.yaml`/`*.apis.yml`
                Parser->>FS: readString(each file)
                Parser->>Parser: parseIncludedContent(recursive)
            else file
                FS-->>Parser: readString(file)
                Parser->>Parser: parseIncludedContent(recursive)
            end
        end
    end
    Parser->>Beans: convert configSnippets -> BeanDefinitions (shared beanIndex)
    Beans-->>Parser: combined BeanDefinitions
    Parser-->>User: return beans
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested labels

7.x

Suggested reviewers

  • predic8
  • rrayst

Poem

🐰 I hop through YAML, follow each include,
I weave the beans where paths once stood,
I skip the loops and normalize the trail,
A tiny rabbit leaving helpful mail. 🥕

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 8.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title 'Add Yaml include list' directly describes the main feature being implemented: YAML include list functionality for recursive include resolution in configuration files.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch yaml-include-list

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@membrane-ci-server
Copy link
Copy Markdown

This pull request needs "/ok-to-test" from an authorized committer.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (3)
annot/src/main/java/com/predic8/membrane/annot/yaml/GenericYamlParser.java (1)

557-578: Consider documenting the shared mutable state in IncludeContext.

The record's includeStack is intentionally shared across all withSourceFile() copies for cycle detection. Since records are typically expected to be immutable, a brief comment clarifying this design choice would help future maintainers.

📝 Suggested documentation
+    /**
+     * Context for resolving include paths during YAML parsing.
+     * <p>
+     * Note: The {`@code` includeStack} is intentionally shared across all instances
+     * created via {`@link` `#withSourceFile`(Path)} to track the include chain for
+     * cycle detection during recursive parsing.
+     */
     private record IncludeContext(Path sourceFile, Path basePath, Deque<Path> includeStack) {
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@annot/src/main/java/com/predic8/membrane/annot/yaml/GenericYamlParser.java`
around lines 557 - 578, The IncludeContext record uses a shared, mutable Deque
includeStack intentionally for cycle detection; add a concise comment inside the
IncludeContext declaration explaining that includeStack is shared across copies
(including those returned by withSourceFile) by design, why it must remain
mutable for include-cycle detection, and that withSourceFile intentionally
preserves the same includeStack instance rather than copying it to avoid
breaking that logic; reference IncludeContext, includeStack, and withSourceFile
in the comment so future maintainers understand this deviation from usual record
immutability expectations.
core/src/main/java/com/predic8/membrane/core/router/IncludeList.java (2)

9-11: Complete the TODO placeholders in Javadoc.

The @description TODO placeholders at lines 10 and 18 should be replaced with meaningful descriptions before merging. This documentation is used for generating user-facing configuration references.

Also applies to: 17-19

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@core/src/main/java/com/predic8/membrane/core/router/IncludeList.java` around
lines 9 - 11, Replace the placeholder Javadoc `@description` TODO entries in the
IncludeList class with meaningful documentation: update the class-level Javadoc
for class IncludeList to describe its purpose (what an include list config
represents and how it's used), and update the second Javadoc block (the one tied
to the include-list element or related getter/setter) to explain the
configuration semantics (what entries mean, expected format, and any
defaults/behavior). Edit the Javadoc comments around the IncludeList class and
its public API (e.g., the class declaration and the include-list
accessor/mutator) to provide concise, user-facing descriptions used for
configuration reference.

15-27: Consider encapsulation improvements for the includes field.

The field lacks an explicit access modifier (defaulting to package-private) and the getter/setter expose the internal list directly, allowing external modification.

♻️ Proposed fix for encapsulation
-    List<String> includes = new ArrayList<>();
+    private List<String> includes = new ArrayList<>();

     /**
      * `@description` TODO
      */
     `@MCChildElement`(allowForeign = true)
     public void setInclude(List<String> include) {
-        this.includes = include;
+        this.includes = include != null ? new ArrayList<>(include) : new ArrayList<>();
     }

     public List<String> getInclude() {
-        return includes;
+        return new ArrayList<>(includes);
     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@core/src/main/java/com/predic8/membrane/core/router/IncludeList.java` around
lines 15 - 27, Make the includes field private and prevent external code from
mutating the internal list: change the field declaration for includes to
private, update setInclude(List<String> include) to assign a defensive copy
(e.g., new ArrayList<>(include) or Collections.emptyList() for null), and update
getInclude() to return either an unmodifiable view
(Collections.unmodifiableList(includes)) or a defensive copy (new
ArrayList<>(includes)) so callers cannot modify the internal state directly;
keep the method names setInclude and getInclude as-is.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@annot/src/main/java/com/predic8/membrane/annot/yaml/GenericYamlParser.java`:
- Around line 557-578: The IncludeContext record uses a shared, mutable Deque
includeStack intentionally for cycle detection; add a concise comment inside the
IncludeContext declaration explaining that includeStack is shared across copies
(including those returned by withSourceFile) by design, why it must remain
mutable for include-cycle detection, and that withSourceFile intentionally
preserves the same includeStack instance rather than copying it to avoid
breaking that logic; reference IncludeContext, includeStack, and withSourceFile
in the comment so future maintainers understand this deviation from usual record
immutability expectations.

In `@core/src/main/java/com/predic8/membrane/core/router/IncludeList.java`:
- Around line 9-11: Replace the placeholder Javadoc `@description` TODO entries in
the IncludeList class with meaningful documentation: update the class-level
Javadoc for class IncludeList to describe its purpose (what an include list
config represents and how it's used), and update the second Javadoc block (the
one tied to the include-list element or related getter/setter) to explain the
configuration semantics (what entries mean, expected format, and any
defaults/behavior). Edit the Javadoc comments around the IncludeList class and
its public API (e.g., the class declaration and the include-list
accessor/mutator) to provide concise, user-facing descriptions used for
configuration reference.
- Around line 15-27: Make the includes field private and prevent external code
from mutating the internal list: change the field declaration for includes to
private, update setInclude(List<String> include) to assign a defensive copy
(e.g., new ArrayList<>(include) or Collections.emptyList() for null), and update
getInclude() to return either an unmodifiable view
(Collections.unmodifiableList(includes)) or a defensive copy (new
ArrayList<>(includes)) so callers cannot modify the internal state directly;
keep the method names setInclude and getInclude as-is.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 18c67e7d-61c0-4c6e-b195-3318ed6720d6

📥 Commits

Reviewing files that changed from the base of the PR and between 1cdece1 and 3aed931.

📒 Files selected for processing (3)
  • annot/src/main/java/com/predic8/membrane/annot/yaml/GenericYamlParser.java
  • core/src/main/java/com/predic8/membrane/core/interceptor/headerfilter/HeaderFilterInterceptor.java
  • core/src/main/java/com/predic8/membrane/core/router/IncludeList.java

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
annot/src/main/java/com/predic8/membrane/annot/generator/IncludeListClassGenerator.java (1)

61-63: Normalize and defensively copy include input.

Line 62 stores caller state by reference and allows null. This can leak mutability or push null-handling downstream. Prefer copy + null normalization.

Proposed patch
                 `@MCChildElement`(allowForeign = true)
                 public void setInclude(List<String> include) {
-                    this.includes = include;
+                    this.includes = include == null ? new ArrayList<>() : new ArrayList<>(include);
                 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@annot/src/main/java/com/predic8/membrane/annot/generator/IncludeListClassGenerator.java`
around lines 61 - 63, The setInclude method currently assigns the caller's List
reference directly to the includes field and allows null; change setInclude in
IncludeListClassGenerator to defensively copy and normalize the input by setting
this.includes to an empty list when include is null and otherwise to a new
ArrayList<>(include) (optionally wrap with Collections.unmodifiableList(...) if
immutability is desired) so caller state is not leaked and nulls are normalized.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In
`@annot/src/main/java/com/predic8/membrane/annot/generator/IncludeListClassGenerator.java`:
- Around line 61-63: The setInclude method currently assigns the caller's List
reference directly to the includes field and allows null; change setInclude in
IncludeListClassGenerator to defensively copy and normalize the input by setting
this.includes to an empty list when include is null and otherwise to a new
ArrayList<>(include) (optionally wrap with Collections.unmodifiableList(...) if
immutability is desired) so caller state is not leaked and nulls are normalized.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: dbe6e8f9-ced3-40d0-8426-6ab996c0add4

📥 Commits

Reviewing files that changed from the base of the PR and between 5bdd15f and 9691e61.

📒 Files selected for processing (2)
  • annot/src/main/java/com/predic8/membrane/annot/SpringConfigurationXSDGeneratingAnnotationProcessor.java
  • annot/src/main/java/com/predic8/membrane/annot/generator/IncludeListClassGenerator.java

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@annot/src/main/java/com/predic8/membrane/annot/yaml/GenericYamlParser.java`:
- Line 77: The root parse currently calls parseYamlFile(grammar, yaml,
IncludeContext.root(), ...) which gives an IncludeContext without
sourceFile/basePath so relative includes resolve against Path.of(".")); modify
GenericYamlParser so that the initial IncludeContext carries the root YAML's
resource path or parent directory (derive from the yaml Resource/URI passed into
parseYamlFile) instead of IncludeContext.root(); create or use an IncludeContext
factory/constructor that accepts a sourceFile/basePath and pass that when
invoking parseYamlFile at the top-level (and apply the same fix where initial
IncludeContext is created in the other parse entry points referenced around
methods/lines covering the 592-609 area), ensuring all relative include
resolution uses the root file's directory as base.
- Around line 145-149: The directory-include branch in GenericYamlParser
currently iterates Files.list(includePath) directly causing nondeterministic
order and therefore unstable bean-N ids (beanIndex shared across
parseIncludedFile). Change the stream handling so you collect the candidate
Paths (apply the existing Filters: Files::isRegularFile and
GenericYamlParser::isApisYaml), sort them deterministically (e.g. by
Path.toString() or getFileName().toString()), then iterate the sorted list and
call parseIncludedFile(grammar, file, includeContext, beanIndex, includePc) for
each; this preserves the current filters and call sites (parseIncludedFile,
beanIndex) but ensures stable ordering.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: df501282-9726-4ae4-9929-4badb587c596

📥 Commits

Reviewing files that changed from the base of the PR and between 9691e61 and c81dcf3.

📒 Files selected for processing (1)
  • annot/src/main/java/com/predic8/membrane/annot/yaml/GenericYamlParser.java

@christiangoerdes
Copy link
Copy Markdown
Collaborator Author

/ok-to-test

Copy link
Copy Markdown
Member

@rrayst rrayst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

superseeded by #2890 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants